Sparse Spectrum Gaussian Process for Bayesian Optimization

We propose a novel sparse spectrum approximation of Gaussian process (GP) tailored for Bayesian optimization (BO). Whilst the current methods provide desired approximations regression problems, it is observed that this particular form generates an overconfident GP, i.e., produces less epistemic uncertainty than original GP. Since balance between predictive mean and variance key determinant to success BO, are suitable BO. derive new regularized marginal likelihood finding optimal frequencies fix overconfidence issue, particularly The regularizer trades off accuracy in model fitting with targeted increase resultant Specifically, we use entropy global maximum distribution (GMD) from posterior GP as needs be maximized. GMD cannot calculated analytically, first Thompson sampling based approach then more efficient sequential Monte Carlo estimate it. Later, also show Expected Improvement acquisition function can used proxy it, thus making further efficient.